With the rapid growth of mobile devices, a number of user behavior logs have been (and are being) generated by users utilizing their devices (e.g., cellular telephones, tablets, and desktop computing devices) to access webpages and/or mobile services. These user behavior logs contain latent intentions and interests of users. Accordingly, there have been attempts to leverage user behavior logs for solving real-world problems in both academia and industry. For instance, predicting a user's next action or behavior is critical for providing enhanced user experiences. Additionally, many content providers (e.g., companies, corporations, service providers, and the like) are interested in whether or not users being presented with the provided content are being presented with the content they desire. Further, content providers are interested in how likely it is that users will perform certain target behaviors, for instance, converting (making a purchase), providing a recommendation, continuing to browse or view, navigating away from the provided content, or the like.
Creating models that approximate a given user's typical browsing, purchasing, and/or content viewing behavior is an important strategy many content providers would like to employ, particularly as it pertains to new customer acquisition. However, such content providers are faced with many challenges surrounding how to analyze and leverage this log-traced data for supporting real-world applications.
Prior attempts have been made to use machine learning techniques to address some of these issues. Most of them, however, require that the input data has a matrix form. Thus, additional effort is required to represent the sequential, log-traced and temporally-arranged data into a matrix form. For instance, some prior attempts to use machine learning techniques to address these issues focus on feature engineering to extract features from the sequential data that are deemed important and represent the extracted features in the form of a matrix. Once in matrix form, such techniques train the model in a supervised manner with specific tasks and labels (e.g., conversion rates and click-through probabilities).
Recurrent Neural Networks (RNNs) have been used for solving various prediction tasks in many areas. For instance, RNNs have been especially useful in Natural Language Processing tasks where RNNs have shown superior performance on machine translation, document classification, and sentiment analysis. However, event sequence data generally contains information such as timestamps, user tags, and session information, which can be useful but is generally not considered in matrix-based machine learning techniques.